Software applications and/or hardware systems have been provided to enable audio, video, and other media content to be edited, manipulated, enhanced, modified, mixed, and arranged to provide a desired media output. For example, such applications/systems may be used to combine and sequence selected portions of video, audio, animation, and/or other media content files—sometimes referred to herein as “clips”—to provide a desired output. Examples of such applications include without limitation Soundtrack® Pro (“STP”) and Final Cut Pro® (“FCP”) available from Apple Inc.
FIG. 1 shows an example of a media editing/mixing interface 100. In the example shown, media clips denoted “Clip 1”, “Clip 2”, “Clip 3”, and “Clip 4” have been arranged along a time or other sequence line indicated by the numbers “0” to “29”, with the start and stop of each numbered interval being indicated by a vertical dashed line. In the example shown, Clip 1 and Clip 3 would begin being rendered at the start of interval “3”; Clip 2 would begin being rendered at the start of interval “7”; Clip 3 would end at the end of interval “15”; Clip 4 would start at the beginning of interval 18; Clip 2 would end at the end of interval “18”; Clip 1 would end at the end of interval “25”; and finally Clip 4 would end at the end of interval “26”. The Clips 1-4 may comprise any media content. For example, Clip 1 might be video content, and Clips 2, 3, and 4 audio desired to be rendered with corresponding portions of video content in Clip 1.
Some media editing/design applications have a higher resolution, i.e., the ability to place a clip at finer gradations along a time or other sequence line, than other such applications. For example, an application focused primarily on audio content, such as STP, may provide the ability to specify start/stop times to a precision of some small fraction of a second (e.g., a 64 bit floating point value in the case of STP), while a video application such as FCP may only support starting/stopping a clip at an indicated frame number or index, typically at a relatively more course granularity, such as 24-30 frames per second (i.e., timeline gradations of 1/30- 1/24 seconds). In some cases, it may be desirable or necessary to represent in a lower resolution application and/or system media content and/or a project created in a higher resolution system, for example to enable an audio track or mix created in STP to be combined in FCP with video edited and/or arranged in FCP to create a combined multi-media project and/or output. However, the start/stop boundaries of clips as arranged in the higher resolution system, for example at a granularity of some small fraction of a second, may not line up neatly with the gradations of the lower resolution system.
FIG. 2 illustrates the potential misalignment of start/stop boundaries when higher resolution media content is represented in a lower-resolution application or system. As used herein the terms “higher resolution system” and “lower resolution system” include without limitation media editing and/or design software and/or hardware and/or combinations thereof. Note that in this sense the higher and lower resolution “systems” may reside in the same physical computer, such as would be the case if each were an application running on the same machine. In the example shown in FIG. 2, a first clip 202 and a second clip 204 have been arranged in a media editing interface 200 of a first, higher resolution application, associated in FIG. 2 with the lower time or sequence line indicated by the numbers “0” through “29”. The clips are desired to be represented in a second, lower-resolution system, represented in FIG. 2 by the upper, lower-resolution frame indexes/numbers (“Frame 0” through “Frame 5”) and the corresponding thick (non-dashed) vertical lines. In this example, there are an even five intervals of the higher resolution system for each interval (i.e., frame) of the lower resolution system. However, in many cases there will not be such an even ratio. As the example shown in FIG. 2 illustrates, the higher resolution content must be represented in the lower resolution system in a manner that enables the lower resolution system to render the content in a manner that (at least most nearly) matches the designer's intent and is within the capabilities of the lower resolution system. In this example, the lower resolution system can only start or stop clips at frame boundaries, and cannot directly render the clips 202 and 204 as arranged in the higher resolution system, because that would require clip 202 to start in the middle of Frame 0, for example. In the example shown in FIG. 2, clip 202 comprises a portion of an underlying media file 206 that includes other content not included in clip 202, as will be discussed further below.
FIGS. 3A-3C illustrate prior art techniques for representing higher resolution media content in a lower resolution system. In FIG. 3A, the clips 202 and 204 have been shifted and truncated to provide shifted clips 322 and 324; but this approach results in the clips starting and stopping later/earlier than the designer intended, potentially resulting in their being out of synch with other/related content. Also, in this example the end of each of clips 202 and 204 (portions “23-25” and “Q-S”, respectively) has been lost. In the approach shown in FIG. 3B, clip 202 of FIG. 2 has been truncated at each end to create a truncated clip 332, which starts and stops on the lower resolution frame boundaries. Likewise, clip 204 has been truncated to create a truncated clip 334 that aligns with the frame boundaries of the lower-resolution system. In this approach, the portions included are rendered at the time originally desired but the truncation results in parts of the media content—in this example portions “13” and “14” at the beginning of clip 202 and portion “25” at the end, as well as portions “G-I” at the beginning of clip 204, being lost (i.e., not represented and therefore not rendered) in the lower-resolution system. Finally, in the approach shown in FIG. 3C, content comprising clips arranged in a higher resolution system is incorporated into one or more clips expanded in length at the beginning and/or end, as required, for example by adding silence or other null content, to create one or more expanded clips that begin and end on frame (or other lower resolution system) boundaries, such as the expanded clip 342 of FIG. 3C. The approach shown in FIG. 3C, however, may limit flexibility, e.g., the ability to further edit or manipulate clips in the destination (lower-resolution) system. Typically a clip such as clip 202 comprises a view or other representation and/or designation of a selected portion of an underlying media content file, such as an audio or video content file, for example the media file 206 shown in FIG. 2. By way of example, the clip 202 may include a selected scene or other portion of an audio or video file that include other content before or after the portion included in the clip. The lower resolution system may allow a clip to be modified (or a new clip defined) to include such additional portions, e.g., by moving the clip start boundary to the left or the clip end boundary to the right, but such flexibility to further edit would not be available if only a static combined and expanded clip such as clip 342 were provided to the lower-resolution system.